Self-organized Reservoirs and Their Hierarchies
نویسنده
چکیده
We investigate how unsupervised training of recurrent neural networks (RNNs) and their deep hierarchies can benefit a supervised task like temporal pattern detection. The RNNs are fully and fast trained by unsupervised algorithms and only supervised feed-forward readouts are used. The unsupervised RNNs are shown to perform better in a rigorous comparison against state-of-art random reservoir networks. Unsupervised greedy bottom-up trained hierarchies of such RNNs are shown being capable of big performance improvements over single layer setups.
منابع مشابه
Reservoir Computing and Self-Organized Neural Hierarchies
There is a growing understanding that machine learning architectures have to be much bigger and more complex to approach any intelligent behavior. There is also a growing understanding that purely supervised learning is inadequate to train such systems. A recent paradigm of artificial recurrent neural network (RNN) training under the umbrella-name Reservoir Computing (RC) demonstrated that trai...
متن کاملSelf-Organization of Surface Transportation Networks
This research investigates the self-organization of surface transportation networks. Using a travel demand model coupled with revenue, cost, and investment models, experiments are run under a variety of parameters on a grid network. It is found that roads, contiguous sections of multiple links operating with similar characteristics, and hierarchies of roads emerge under a broad range of assumpt...
متن کاملGuided Self-Organization of Input-Driven Recurrent Neural Networks
We review attempts that have been made towards understanding the computational properties and mechanisms of input-driven dynamical systems like RNNs, and reservoir computing networks in particular. We provide details on methods that have been developed to give quantitative answers to the questions above. Following this, we show how self-organization may be used to improve reservoirs for better ...
متن کاملAbstraction Hierarchies for Engineering Design
Complex engineering design problems consist of numerous factors of varying criticalities. Considering fundamental features of design and inferior details alike will result in an extensive waste of time and effort. Design parameters should be introduced gradually as appropriate based on their significance relevant to the problem context. This motivates the representation of design parameters at ...
متن کاملStatistical Prediction of Probable Seismic Hazard Zonation of Iran Using Self-organized Artificial Intelligence Model
The Iranian plateau has been known as one of the most seismically active regions of the world, and it frequently suffers destructive and catastrophic earthquakes that cause heavy loss of human life and widespread damage. Earthquakes are regularly felt on all sides of the region. Prediction of the occurrence location of the future earthquakes along with determining the probability percentage can...
متن کامل